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Abstract 

We study the phenomenon of loss of lock in the optimal non-causal 
phase estimation problem, a benchmark problem in nonlinear estima- 
tion. Our method is based on the computation of the asymptotic 
distribution of the optimal estimation error in case the number of 
trajectories in the optimization problem is finite. The computation 
is based directly on the minimum noise energy optimality criterion 
rather than on state equations of the error, as is the usual case in 
the literature. The results include an asymptotic computation of the 
mean time to lose lock (MTLL) in the optimal smoother. We show 
that the MTLL in the first and second order smoothers is significantly 
longer than that in the causal extended Kalman filter. 
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1 Introduction 

In many applications in communication practice a random signal x{t) is received 
through a noisy channel. The random signal x{t) G is assumed to be a stochastic 
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process defined by an Ito stochastic differential equation (SDE) [T7j 



dx = m{x, t) dt + cr(x, t) dw, 



(1) 



where ti'(t) is a vector of standard Brownian motions. The measurements process 
y{t) G M^, which is the output of the noisy channel, is modeled by another SDE 



where Nq measures the channel noise intensity and v{t) is another vector of Brownian 
motions, independent of w(t). We further assume that the functions m(-, ■), cr(-, ■) 
and g{-,-) satisfy standard regularity conditions such that ([I]),© possess a strong 
unique solution. 

When the optimality criterion is minimum square error, the optimal filtering prob- 
lem is to construct the causal estimator x{t) = E [x(t) \ y{s)] of x(t), where < s < t 
[32] . The optimal fixed interval smoothing problem is to construct the non-causal es- 
timator x{t) = E [x{t) I y{s), < s < T], where T is the length of the interval, and 
t < T. The optimal fixed lag smoothing problem is to construct the non-causal esti- 
mator x{t) = E [x{t) \ y{s) , < s < t + A], where T is the length of the interval, and 
t + A < T. In many applications delay in the estimation of the signal is not permis- 
sible, as for example in closed-loop control, radar tracking systems, and so on. There 
are, however, interesting cases, where certain delay is permissible, as for example in 
communication systems, as extensively practiced in coding [3^ [20]. 

Smoothers are used because their performance is superior to all causal filters, with 
respect to the same optimality criterion p^. In linear estimation theory, where the 
optimality criterion is minimum mean square error, the error variance of the optimal 
smoother is smaller than that of the optimal filter |10j . 

Optimal estimators are usually infinite- dimensional and therefore have no finite- 
dimensional realizations, so that suboptimal estimators have been proposed to approx- 
imate the optimal ones by a system of SDKs, driven by the measurements [121 EE [22] • 
The phase-locked- loop (PLL), which is a realization of the extended Kalman fil- 
ter (EKF) [30], is a nonlinear suboptimal causal estimator of the carrier phase in 
various communications systems [16]. A well known effect in such PLL demodula- 
tors is the cycle slip phenomenon that consists in occasional sudden changes of size 
271 n [n = ±1, ±2, . . . ) in the phase estimation error [5J. Obviously, the mean time 
between cycle slips, known as the mean time to lose lock (MTLL), decreases with the 
noise intensity and causes sharp degradation in the performance of the filter and to 
the formation of a performance threshold [331 l29l 128] . 

Considerable effort was put into the computation of the MTLL in causal estima- 
tors [ini [Ml [SB [S3] , including the singular perturbation method [2E1 [2SJ and large 
deviations theory [51 [S]- However, the phenomenon of loss of lock in smoothers has 
never been addressed, despite the extensive study of linear and nonlinear smoothers 
in the hterature [2ll[22l[23l[Tl[35l[6l[2l[25l[2ni[l0l[T5in^ The objective of this 
paper is to provide the missing theory, estimate the MTLL in the optimal smoother, 
and compare it with that in the casual PLL. Specifically, we compute the asymptotic 
distribution of the optimal estimation error in case the number of trajectories in the 



dy = g{x, t) dt + dv 



(2) 
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optimization problem is finite. We identify tlie contribution of error trajectories to 
the minimum noise energy (MNE) cost functional and recast the problem in terms of 
order statistics. The asymptotic expression for the MTLL in the smoother is similar 
to that resulting from the Wentzell-Freidlin theorem for causal systems, with a new 
functional. Applying our method to standard phase models, we show that the MTLL 
in the optimal smoother is significantly longer than that in the PLL. 

2 The mathematical model 

The general equations of a scaled phase tracking system consist of the linear model 
of the phase x{t) = X2(t), Xiv(t)]'^ [29] 



X = Ax + ^/eBw (3) 
and the nonlinear model of the noisy measurements yit) = [ys{t),yc{t)]'^ 

y = h{x) + ^v, (4) 



with 



hix) 



smx 
cosx 



The dimensionless parameter e is assumed small in the case of a low noise channel 
[28]. 

A fixed-interval minimum noise energy (MNE) estimator x{-) for x{-) is the min- 
imizer of the cost functional [6] 

J[z{.)] = f [\y-h{z)\' + \C\']dt, (5) 
Jo 

with the equality constraint 

z = Az + BC, (6) 

that is, 

= argmin J[2;(-)]. (7) 
•^(■) 

Note that the integral J[a3(-)] contains the white noises w{t), v{t), which are not 
square integrable. To remedy this problem, we begin with a model in which the white 
noises w(t), v(t) are replaced with square integrable wide band noises, and at the 
appropriate stage of the analysis, we take the white noise limit (see below). 

In contrast to nonlinear filtering, where the locked state is a local attractor for the 
error dynamics [29] , and escaping it corresponds to loss of lock, there is no dynamics, 
and therefore no attractors for smoothers. Thus, we have to define cycle slip events in 
a different manner than hitting the boundary of the domain of attraction. We define 
the estimation error e{t) = [e(t), e2(t), . . . , e^it)]'^ as 

e{t) = x{t) - x{t), (8) 
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Figure 1: An example of two trajectories in C^(to = 5) 

and we say that a cycle-slip has occurred in the time interval (to, to + At), At « 1, 
if and only if the estimation error e{t) vanishes at at some to — Ti, reaches the 
point [27171,0'" , {n = ±1, ±2, ...), at some later time to + T2, and e(t) = nn, where 
t G (to,to + At). The time Tg = Ti + T2 is the slip duration, satisfying Tg « T. 
We define in the space of continuous functions ^[^^-j the set ^^(to) of all continuous 
trajectories e(-) that slip in the interval (to, to + At). Thus 

Pr {slip in (to, to + At)} = Pr {e{-) G <^^(to)} . (9) 

For small values of e the cycle-slip event is a rare large deviation from the original 
trajectory, and therefore x{t) is in the vicinity of x{t) before the cycle-slip, and in 
the vicinity of x{t) + [27m, 0]^ after the slip. Thus, we can define the beginning and 
the end of the cycle-slip by the instants when e(t) reaches the origin and [27rn,0]"^, 
respectively. An example of two trajectories in ^^(tg = 5) is given in Figure [H 

3 The MTLL in the optimal smoother 

The Wentzell-Freidlin theorem [S], [H] and the singular perturbation method [29] for 
asymptotic evaluation of the MTLL are concerned with stochastic processes satisfying 
a stochastic differential equation with a unique solution. In contrast, the dynamics 
of the optimal smoother, derived from the EL equations, form a two-point boundary- 
value problem which has no unique solution. Therefore the Wentzell-Freidlin and the 
singular perturbation method seem inappropriate for the computation of the MTLL 
in a smoother. It appears that this computation calls for a different approach. 



The first step toward an asymptotic calculation of the MTLL in smoothers is the 
computation of the asymptotic distribution of the estimation error in case the number 
of trajectories in the optimization problem is finite. We investigate the cost functional 
of deterministic error trajectories that deviate from the original trajectory x{t). We 
augment x{t) with the set of the A^^^ trajectories rj(t) = [rj(t), rf (t), . . . ,rl^\t)] G 
'^[Qj'j, i & [1; • • • ; Nt]- The trajectories x{t) + ri(t) are admissible in the optimization 
problem ([5]), ([6]), only if the trajectories Viit) satisfy 



Vi = Avi + Bui. 

We define the difference 

AJ[=r(-),r,(-)] = J[=r(-) + r-.(-)]-J[=^(-)] 
and substitute ([5]), ([6]) and ([10]) in ([II]) to obtain 



(10) 
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AJ[a;(-),r,(-)] 



4sinM^) + l^^.'' 



dt 



+ y^e I [sina; — sin(x + Tj)] (ifi(t) 

+ v4£ / [cosx — cos{x + ri)]dv2{t) 
Jo 

+ \f^e [ ujdw{t). 
Jo 



:i2) 



At this point, we take the white noise limit in the wide band noises so the stochastic 
integrals in ([T^ become Ito integrals. Collecting them into a single Ito integral leads 
to 



AJ[x{.),r,{.)] 



Asm' ' ^ ) + I""*! 



dt 



(13) 



+ VAe 



4sin^ - + l""*!^ dvi{t), 



where Vi{t) is a standard Brownian motion that depends on v{t) and w{t). 

Note that although the values of the random variable AJ[x{-),ri{-)] depend 
on the trajectories of x{t) and y{t) through v{t) and w{t), the probability law of 
AJ[x{-),ri{-)] depends only on the trajectories r'j(t). Therefore, we abbreviate nota- 
tion to AJ[rj(-)]. We note further that AJ[rj(-)] is a Gaussian random variable with 
expectation 

EAJ[r,(-)]=m„ (14) 
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where 



and variance 



rrii 







4sin^ (-^] + \u, 



dt. 



Var{AJ[r,(-)]} = Aerrii. 
Furthermore, we compute the covariance 

a,, = E(AJ[r,(-)] -m,) (AJ[r,(-)] -m,) 

r-T pT 



(15) 
(16) 



Ae I ujuj dt + Ae [1 + cos(rj — rj) — cosr^ — cosr^] dt. 
Jo Jo 



(17) 



Note that if the supports of rj(-) and rj{-) are disjoint, the cost functionals AJ[rj(-)], 
AJ[rj{-)] are not correlated. We conclude that the random variables AJ[rj(-)], i = 
1 . . . Nt, form a Gaussian random vector with distribution 

AJ[n(.)] 
A^[^2(-)] 



AJK,(-)] 







mi 




mi 


Cri2 
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m2 
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where o"ij = —a,:,-. 



Ae 



When considering a finite number of error trajectories rj(-), i = 1,...,Nt in 
the optimization problem. The estimator error trajectory eAfj,(-) minimizes the cost 
functional AJ[rj(-)], 



Pr {bnj 



Vki-)} = Pr {AJ[rk{-)] < AJ[r,(-)] for all j k} . 



(19) 



Thus the problem of minimization has been recast in the language of order statistics. 
The probability on the right side of (fT9l) is difficult to calculate, so we pursue the 
distribution of the error trajectory ejVy(-) in the limit of small e. We assume that for 
each k there exists an interval Aj. such that 

E{AJ[r,(-)]|AJ[n.(-)G A,]}> AJ[rfc(-)] for all j ^ k. (20) 

Cramer's theorem for Gaussian vectors [S], [7| implies that in the limit of small e 

lim£log,Pr{AJ[rfc(-)] < AJ[rj(-)] for all j ^ k} = 

(21) 

limelog,Pr{AJ[rfc(-)] G Ak} . 

e^O 

Next, we evaluate the conditional expectation 

E{AJ[r,(-)]|AJ[r,(.)]}. (22) 
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Since AJ[rj(-)], AJ[rk{-)] are jointly Gaussian, 



E{AJ[r,(.)]|AJK(.)]} 



rrij + — 
Aemk 



(AJ[rfc(-)] -mfc) 



(23) 



m, + ^(AJ[r,(-)] 



rrik] 



In order to determine the interval A^, defined in f l20|) . we derive the set of Nt — 1 
inequalities 

E {AJ[r,(.)] I AJ[r,(-)]} > A J[r,(-)] for all j ^ A;. (24) 
The interval A^, which is the range of values of AJ[rfc(-)] satisfying ( l24l) is defined by 



max 
r,{.)GA- 



< AJ[rk{-)] < min 



rrii 



Ojk_ 

rrik 



Ojk 

rrik 



(25) 



where A'^ is the set of all trajectories rj{-) such that rnj < ruk, (Jjk > rrij, and A~ is 
the set of all trajectories rj{-) such that rrij > rUk, CFjk > rUk- Note that the supremum 
and infimum, over all continuous trajectories, of the leftmost and rightmost sides of 
(!25|) . respectively, is —rrik. This means that as Nt increases and the trajectories 
x{-) + rj(-) are sampled from ^[o,t] according to their a priori distribution ([3]), the 
interval Ak narrows. Specifically, for any 6, 6 > there is a sufficiently large Nt such 
that Pi {Ak ^ (— — 5, — + 5)} < 6. Combining f|T9|) and fl2Tl) . we conclude 
that for all small e > and every sufficiently small 6, such that < 6 < e, there is a 
sufficiently large Nt such that 



Pr {eiVj 



Pr {-mfc - 5 < AJ[rfc(-)] < -rrik + S} 
25exp 



Ami 



6e ruk 



(26) 



2(5 exp 



1 

'2i 



. . 2 fr-k 
4 sm ' 



+ \Uk\ 



dt 



Based on the distribution of the estimation error ejVr(-) in the case of a finite 
number of error trajectories Nt (1261) . the probability that e.N^{-) is in any set in 

[0,T] 



•^S^. is 



Pr{e;v^(-) e ^} 



^ Pr{ejvj,( 



25 exp 

rfe(-)e.c/ 



n(-)} 



1 

2^ 



4sin^ 



2 /^fc 



life 



(27) 
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Applying Laplace's method for sums of exponentials with large parameter - |3], we 
obtain the asymptotic expression 



Pr |e7v„(-) G X exp < — — min 



4 sm — 
2 



\Uk\ 



dt 



{2i 



In the limit 5^0 (and Nt —>■ oo), the probability that the optimal estimation 
error e(-) is in is found as 

Pr |e(-) G i^l X exp inf [ 

I 2£r(.)e^7o 



4 sin (-] + \u 



dt 



(29) 



subject to the equality constraint 



Ar + Bu, 



(30) 

G £/. Note 



where the infimum in fl29l) is taken over all continuous trajectories r 
that the small e approximation is taken before the limit 6-^0. 

We turn now to the computation of the MTLL. First, we consider time intervals 
[0, T] much longer than the time constant of the system, so that most of the time 
the system is in steady state. It follows that the probability Pr {slip in (to^^o + ^t)} 
is independent of to for to outside intervals of fixed length (the time constant of the 
system) at the endpoints and T. Therefore, in view of the regularity of the pdf of 
the solution as a function of t and the independence of cycle slips in disjoint intervals 
(see above), for such tg 

Pr{slip in (to, to + 2 At)} = 



Pr {slip in (to, to + At)} + Pr {slip in (to + At, to + 2 At)} + o(At) 



(31) 



2 Pr {slip in (to, to + At)} + o(At). 

Thus, Pr {slip in (to, to + At)} is nearly linear in At. 

Next, we note that for fixed At fl29p implies that the slip probability satisfies 



Pr {slip in (to,to + At)} 
Pr{e(-)G^^(to)} 



(32) 



exp 



— inf , 

2e r(-)&v^{to) Jo 



4sin^ 



+ u 



dt 



subject to the equality constraint (130|) . Equations (!3T|) and (l32l) can be written 
together as 



Pr{slip in (to,to + At)} 



(At + o(At))Q(e)exp{- inf — , 



4 sin 



\u\ 



dt 



(33) 
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subject to the equality constraint (150]) . where e log fl{e) — > as e ^ 0. 

Equipped with the slip probability (1331) . we turn to the evaluation of the MTLL 
in the optimal smoother. We introduce a renewal (counting) process {^{t), t > 0}, 
a nonnegative integer-valued stochastic process that counts the number successive 
cycle-slip events in the time interval (0, t] [11]. We assume that the time durations 
between consecutive slips are positive, independent, identically distributed random 
variables. Based on the above assumptions, we adopt the renewal formula [TT] 

_ t 

"""^ " E^fXt) ' ^^^^ 
where r„c is the MTLL in the non-causal estimator. In the limit of t — oo, equation 



gives 



lim —— 



lim : , 



(35) 



Using the slip probability fl33|) we obtain 

E^(t) : 

iil— >U 

Pr{slip in {to,to + At)} 



At^O I At J 



lim 



exp 



At 



inf — , 



(36) 



4 sin 



+ u 



dt 



Substituting fl36l) in fl35|) . we obtain the expression for the asymptotic MTLL, r„c, in 
the optimal MNE estimator 

r-T 



lirn 6 loge Tnc 



inf 



e^o e(-)e^«f^{to) 2 Jo 

subject to the equality constraint 

e = Ae + B^. 



4 sin^ - 



(it, 



(37) 



(38) 



4 The MTLL in the smoother with standard phase 
models 

We begin with the first order phase tracking system suggested by and [18], in 
which the phase x{t) is modelled as a standard Brownian motion 



X 



w 



y = h{x)+pv, 



(39) 
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where x(t),w(t) take values in M}. The system ( 15^ is scaled to the form of ([3]), (jl]) 
with A = 0, B = 1 and e = p. A similar procedure is presented in [1]. Note that the 
CNR equals p-'^/2 in the system fl39|l . 

In the first order case, the asymptotic expression for the MTLL in the smoother 
( l37|l becomes 



limeloggTnc 



inf 









r 


4 sin^ ( 




/o 


L V 





(it, 



(40) 



The variational problem on the right side of fl40p is solved analytically using the 
Hamilton- Jacobi-Belman equation [12] . This leads to the asymptotic limit of the 
MTLL in the first order non-causal estimator as 



lim e logg Tnc = lim p logg r„c = 8. (41) 

e— >0 p— >0 

In order to compare the MTLL in the optimal smoother fl4ip with that in the 
suboptimal PLL we construct the steady-state EKF corresponding to the model 0391) 

X = — [ys cos X — Uc sin x) . (42) 

The differential equation of the causal EKF estimation error e{t) = x{t) — x{t) is 
scaled to the form [3] 

e = — sine -I- -\/2et'- (43) 
The asymptotic MTLL, r^, in this simple analytical potential case is known to be [29] 

lim e logg Tc = lim p logg Tg = 2. (44) 

e— >0 p— >0 

Note that in first order estimators the MTLL in the non-causal estimator pil) 
is significantly longer than that in the causal estimator (jH]). This implies that the 
CNR values in the smoother are smaller than in the filter, but give a MTLL identical 
to that in the filter. Denoting by eg, 5„g the values of e in the filter and smoother, 
respectively, and requiring identical MTLLs, lead to 

rc = Tnc^ — = — • (45) 

Denoting by CNRc[dB], CNR„c[dB] the CNR in the filter and smoother, respectively, 
and using the simple relation between e and the CNR, lead to 

CNRg[dB] - CNR„g[dB] = 10-2 logio(8/2) ^ 12dB. (46) 

Thus, there exists a 12dB performance gap in the MTLL between the estimators in 
terms of CNR. The MTLL in first order non-causal MNE smoother and causal EKF 
are given in Figures [2] and [3l The pre-exponential term in the plots is arbitrary. 
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Figure 2: The MTLL in the first order optimal MNE estimator and the causal 
PLL as a function of 
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CNR[dB] 

Figure 3: The MTLL in the first order optimal MNE estimator and the causal 
PLL function of the CNR. 
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Next we consider the more complex, and more realistic, case of a second order 
phase model [30], in which the phase is modelled as an integral over a Brownian 
motion. The signal model is 

X = A'x + B'w, (47) 

where 

The measurements model is 

y = h{x)+pv, (48) 

where x{t) and w{t) take values in R^. The system (H7!) . psj) is scaled to the form of 
([SD,® with A = A', B = B', ande = p^/^. Note that in this case the CNR equals 
p~^/2, as in the case of the first order system. 

In the second order case, the asymptotic expression for the MTLL in the smoother 
f l37|) becomes 

\ime\og^Tnc= inf - / 4sin^(-)+e^ dt, (49) 



variational problem on the right side of fl371) as no analytical solution. An ap- 
proximate solution is obtained numerically about the characteristics of the Hamilton- 
Jacobi-Belman equation [29l [5] . This leads to the asymptotic limit of the MTLL in 
the second order non-causal estimator as 

lim e loge Tnc = lim p^^^ logg r„c = 5. (50) 

We note that the EKF corresponding to the second order system (H71) . (HHj) has 
the error equations [30].[5] 



1 , — 

e = -Lf — sine — v2e' i) 

2 (51) 



(p = — sin e — y/2e^ i) + V2e' w 



where e' = e/^/2. The "eikonal" equation j.5j for the quasi-potential $ corresponding 
to §2) is 

- sin -l-e - sin e% + <i>l + 2$^$^ + 2<l>2 = 0. (52) 

The function $ is evaluated numerically on the characteristics of ( l52l) [5]. This 
procedure leads to the value of the quasi-potential at the unstable equilibrium point 
$(</) = 0, e = tt) = 0.6, so the asymptotic MTLL in the second order causal estimator 
is 

lim e logg Tc = lim p^^^ logg Tg = 0.85. (53) 
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Note that similarly to the first order case, the MTLL in the second order smoother 
fl50|) is significantly longer than that in the causal PLL fl53|) . The CNR gap in this 
case is 

CNRc[dB] - CNR„4dB] = 40/3 logio(5/0.85) ^ 10.25dB. (54) 

The MTLL in the non-causal estimator and the causal EKF in the second order system 
are given in Figures H] and O The pre-exponential term in the plots is arbitrary. 



5 Discussion and conclusions 

The significant advantage of the optimal smoother over the causal PLL defies intu- 
ition. It was customary to think that the MTLL advantage of the smoother is linked 
and proportional to the MSE advantage of the smoother [32]. We argue that the 
MSE in an estimator is not related to the MTLL. In order to demonstrate this idea 
we introduce the following error equation 

g 

e = —2sm- + ^/ev. (55) 

The MSE in the linearized version of fl55|) is identical to that in linearized version of 
the system 

e = — sine + y/ev. (56) 
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Figure 5: The MTLL in the second order optimal MNE estimator and the 
causal PLL as a function of CNR. 



However, due to the difference in the potential barrier in the above systems, the 
asymptotical MTLL, ri, in the system (155]) satisfies 



1 „ r' ^ e\2 



lim e log, Ti = - inf / ( e + 2 sin - ) dt = 8, (57) 

6^0 2 {e(0)=0, e(T')=2^} Jq \ 2/ ' ^ ' 

while the MTLL, r2, in the system fl56l) satisfies 

limeloggr2 = 4, (58) 

and is significantly shorter than ri. Thus, we conclude that the MTLL advantage 
of the optimal smoother over the PLL is due to an entirely different functional and 
cannot be predicted by the MSE advantage of the smoother over the filter. 

There is a fundamental mathematical difference between the causal and the non- 
causal cases. Both problems involve the minimization of a functional, similar to that 
of the Wentzell-Freidlin theory for causal systems. There is, however, a difference 
between the functionals in the two theories. While the functional in the causal case 
vanishes along the exiting trajectories from the boundary of the domain of attraction 
of the locked state to the next locked state [9j, in the non-causal case the functional 
vanishes only at the locked states, so it has to be computed along the entire slip 
trajectory. 
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